Adaptive interface device that is programmable and a system and method of programming an adaptive interface device

ABSTRACT

An adaptive interface device that is programmable and a system and method of programming an adaptive interface device. To program the adaptive interface device, a user navigates navigable data structure stored on the adaptive interface device through certain combinations of user inputs using the programmable inputs. The adaptive interface device outputs navigation codes indicative of the user&#39;s navigation of the navigable data structure. A connected computer receives and interprets the navigation codes and provides visual feedback to a user to assist in programming. More particularly, a web browser of the connected computer receives the navigation codes and provides the user with a graphical representation of the navigable data structure and the user&#39;s navigation thereof.

RELATED APPLICATIONS

The application claims priority to U.S. Provisional Patent ApplicationNo. 62/047,618, filed Sep. 8, 2014, the entire contents of which areincorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to adaptive interface devices.

BACKGROUND

Personal computing devices, including, for example, desktop computers,laptop computers, tablets, smart phones, and personal digitalassistants, typically include or can be coupled to various interfacedevices, such as a keyboard, mouse, and gaming controller. Generally,some external input (e.g., a human touch or movement) causes theinterface device to output predetermined data or control signals to thecomputing device. The data or control signals are then received bysoftware executing on the computing device, which cause the computingdevice to react in a manner in accordance with the software.

Some interface devices are adaptive and may be configured by a user toalter the particular outputs generated in response to particular inputson the interface device. Nevertheless, configuring an adaptive interfacedevice may require specialized software executing on the computingdevice and bi-directional communication with the adaptive interfacedevice to enable the computing device to overwrite data on the adaptiveinterface device.

Accordingly, there is a need for improved systems and methods forprogramming of interface devices.

SUMMARY

In one embodiment, the invention provides a method of programming anadaptive interface device. The method includes providing a remappinggraphical user interface on a display screen of a computing device andproviding, on the remapping graphical user interface, a graphicalrepresentation of a navigable data structure of the adaptive interfacedevice. The adaptive interface device receives navigation codesindicative of navigation inputs. The graphical representation is updatedbased on the navigation codes. The method includes determining based onthe navigation codes that the adaptive interface device has beenremapped to have an updated mapping. The updated mapping of the adaptiveinterface device is displayed on the remapping graphical user interface.

In some instances, the computing device receives a data cable coupled tothe adaptive interface device, wherein the navigation codes are receivedover the data cable. In some instances, the method includes receiving aremap mode message from the adaptive interface device, where the remapmode message indicating that the adaptive interface device has entered aremap mode. In some instance, the method includes receiving currentmapping data from the adaptive interface device and displaying, on theremapping graphical user interface, a current mapping of the adaptiveinterface device based on the current mapping data. In some instances,the method includes displaying, on the remapping graphical userinterface, a virtual adaptive interface device including programmableinputs of the adaptive interface device. In some instances, displayingthe updated mapping of the adaptive interface device includes displayingthe updated mapping on the virtual adaptive interface device. In someinstances, the graphical representation of the navigable data structureis a virtual keyboard.

In another embodiment, the invention provides another method ofprogramming an adaptive interface device. The method includestransmitting, by the adaptive interface device, a remap mode message toa computing device. The remap mode message indicates that the adaptiveinterface device has entered a remap mode. The adaptive interface devicereceives navigation input that navigates a navigable data structure ofthe adaptive interface device used to remap programmable inputs of theadaptive interface device. The adaptive interface device remaps theprogrammable inputs based on the navigation input. The method furtherincludes transmitting, by the adaptive interface device, navigationcodes indicative of the navigation input received and indicative of theremapping.

In some instances, the adaptive interface device receives a data cablecoupled to the computing device, wherein the navigation codes aretransmitted over the data cable. In some instances, the method includes,upon entering the remap mode, transmitting current mapping data to thecomputing device, where the current mapping data is indicative of acurrent mapping of the programmable inputs of the adaptive interfacedevice. In some instances, the method includes receiving a request, viathe programmable inputs, to enter the adaptive interface device into aremap mode. In some instances, the method includes receiving user inputat a first programmable input of the programmable inputs beforereceiving the request to enter the remap mode. These instances mayfurther include transmitting, by the adaptive interface device, a firstcode to the computing device representing a first mapped output inresponse to the user input; receiving further user input at the firstprogrammable input after the remapping; and transmitting, by theadaptive interface device, a second code to the computing devicerepresenting a second mapped output in response to the further userinput.

In another embodiment the invention provides a programmable adaptiveinterface device. The programmable adaptive interface device includesprogrammable inputs, an input/output interface, a memory, and aprocessor. The input/output interface is configured to be coupled to acomputing device. The memory includes a navigable data structure and akey mapping index. The processor is configured to transmit a remap modemessage via the input/output interface to the computing device, wherethe remap mode message indicates that the adaptive interface device hasentered a remap mode. The processor is further configured to receive,via the programmable inputs, navigation input that navigates thenavigable data structure. The processor remaps the programmable inputsbased on the navigation input and transmits, via the input/outputinterface to the computing device, navigation codes. The navigationcodes are indicative of the navigation input received and indicative ofthe remapping.

In some instances, the programmable adaptive interface device includes adata cable coupled to the input/output interface and the computingdevice to form a communication link. In some instances, the key mappingindex relates each of the programmable inputs to a corresponding outputcode. In some instances, upon entering the remap mode, the key mappingindex has current mapping data indicative of a current mapping of theprogrammable inputs of the adaptive interface device. In some instances,upon completion of the remapping, the key mapping index has updatedmapping data indicative of an updated mapping of the programmable inputsof the adaptive interface device. In some instances, the processor isfurther configured to, receive user input at a first programmable inputof the programmable inputs; transmit a first code to the computingdevice representing a first mapped output in response to the user input;receive further user input via the first programmable input after theremapping; and transmit a second code to the computing devicerepresenting a second mapped output in response to the further userinput. In some instances, the device further includes a conductorselectively secured to two of the programmable inputs to form aconductive path between the two programmable inputs. The processor isfurther configured to detect the conducted path formed by the conductorand, in response, enter the adaptive interface device into the remapmode.

Other aspects of the invention will become apparent by consideration ofthe detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates a front side of an adaptive interface device inaccordance with some embodiments of the invention.

FIG. 1B illustrates a back side of the adaptive interface device of FIG.1A.

FIG. 2 illustrates a diagram of the adaptive interface device of FIG. 1Aoperating as an interface device for a computing device.

FIG. 3 illustrates a diagram of a reprogramming system including theadaptive interface device of FIG. 1A.

FIGS. 4A and 4B illustrate methods for programming an adaptive interfacedevice, such as the adaptive interface device of FIG. 1A.

FIGS. 5A, 5B, 5C, 5D, and 5E illustrate various screens of a remappinggraphical user interface in accordance with some embodiments of theinvention.

DETAILED DESCRIPTION

Before any embodiments of the invention are explained in detail, it isto be understood that the invention is not limited in its application tothe details of construction and the arrangement of components set forthin the following description or illustrated in the following drawings.The invention is capable of other embodiments and of being practiced orof being carried out in various ways.

FIGS. 1A and 1B illustrate a front and back view, respectively, of anadaptive interface device 100 in accordance with some embodiments. Theadaptive interface device 100 is a programmable device that simulates acomputer peripheral device, such as a keyboard or mouse, based on userinputs. The user inputs include, for instance, actions that result incompleting a conductive circuit loop. The user inputs are detected bythe adaptive interface device 100, which, in turn, generates outputsthat simulate peripheral device outputs.

The adaptive interface device 100 includes a printed circuit board (PCB)102 and a connector cable 104 (e.g., a universal serial bus (USB®)cable). The PCB 102 includes a microcontroller 106 executing firmwarestored on a local memory of the microcontroller 106 or another memory ofthe PCB 102. The PCB 102 includes a port 108 that receives the connectorcable 104.

With reference to FIG. 3, the opposite end of the connector cable 104 iscoupled to a peripheral port 110 of a computer 112 (e.g., a desktopcomputer, a laptop, or tablet). The connector cable 104, port 108, andperipheral port 110 may be, for example, one of a USB®, Firewire®, orThunderbolt® port. The PCB 102 receives power (e.g., 5 volts directcurrent (DC)) from the computer 112 via the connector cable 104, whichpowers components of the PCB 102 including the microcontroller 106. ThePCB 102 uses the Human Interface Device (HID) protocol to communicatewith the computer 112 over the connector cable 104. The communicationssignal to the computer 112, for example, particular keyboard (key)presses, mouse clicks, and mouse movements. In some embodiments, inplace of or in addition to the connector cable 104, a wirelessconnection, such as a Bluetooth® or Wi-Fi®, is used for communicationsbetween the PCB and the computer. In these wireless implementations orimplementations where a wired connection does not provide power, aportable power supply (e.g., a battery) may be coupled to the PCB 102 toprovide power.

Returning to FIGS. 1A and 1B, the PCB 102 includes several input pinsthat are coupled to the microcontroller 106 (e.g., via traces on thePCB). The PCB 102 includes six earth/ground pins 114 provided along thebottom. Additionally, the PCB 102 includes a plurality of input pins,referred to generally as input pins 116, but more particularlyidentified herein with a letter appended to the identifier 116 (e.g., uparrow pin 116 a). The front of the PCB 102 includes six input pins 116:the up arrow pin 116 a, down arrow pin 116 b, left arrow pin 116 c,right arrow pin 116 d, space pin 116 e, and click pin 116 f,collectively referred to as input pins 116. Each ground pin 114 andinput pin 116 includes two conductive apertures, which enable quickconnections of alligator clip cables because each half of the alligatorclip is received by a respective aperture, and the (spring-loaded)alligator clip clamps onto the bridge portion of the PCB dividing thetwo apertures. Of course, conductors without alligator clips may becoupled to the input pins as well, such as via soldering and othertechniques.

The back of the PCB 102 (FIG. 1B) has several additional input pins foradditional keyboard keys and mouse controls. As illustrated, the back ofthe PCB 102 includes eight additional keyboard inputs 116 g for keyboardkeys, which are individually labeled W, A, S, D, F, G, H, and J, and sixadditional input mouse pins 116 h for mouse controls, which aregraphically labeled with a mouse up, mouse down, mouse left, mouseright, right click, and left click graphic. The keyboard input pins 116g and mouse input pins 116 h are female headers that may receive paperclip ends, wire ends, jumper ends, or other conductors. The back of thePCB 102 also includes an area for using the board to control outputs.Furthermore, as shown in FIGS. 1A and 1B, because the input pins 116 onthe front side of the PCB 102 include apertures that extend through thePCB 102, these input pins 116 are also accessible for alligator clipconnections from the back side of the PCB 102.

The particular pin layout and number of pins on the PCB 102 areexemplary. In some embodiments, the input pins 116 are located on otherportions of the PCB 102 and/or in other layout arrangements. In someembodiments, the PCB 102 includes more or fewer input pins 116 on thefront, back, or both.

Each of the input pins 116 noted above includes a default pin assignmentsuch that each input pin 116 is mapped to particular key board press(e.g., a “w”), mouse control (e.g., a right mouse click), or other HIDprotocol signal. Labels, graphics, and the layout of the PCB 102 makethe default pin assignment apparent to a user. For instance, the uparrow input pin 116 a is positioned within an up arrow illustration andspace input pin 116 e includes the text “space” beneath it (see FIG.1A). In some embodiments, different default key presses and mousecontrols are assigned to the inputs pins 116. Furthermore, as discussedin detail below, the assignments to the input pins 116 may bereprogrammed such that they map to and simulate different key presses,mouse controls, or other HID protocol signals.

Referring back to FIG. 2, a ground conductor 120 and an input conductor122 are coupled to the ground pin 114 and one of the keyboard input pins116 g, respectively. The keyboard input pin of FIG. 2 is labeled 116g-w, and represents the keyboard input pin 116 g having a default pinassignment of the letter “w” (see uppermost input pin 116 in FIG. 1B).As an example, the ground conductor 120 and input conductor 122 arealligator clip cables, which are wires having alligator clips on one orboth ends. An insulating wrap may be positioned around the wire, exceptfor the ends that are exposed for conductive coupling. The groundconductor 120 and input conductor 122 are coupled at one end to thegroup pin 114 and keyboard input pin 116 g-w, respectively, and, at theopposite end, to a person 124 and an apple 126, respectively. Althoughremovable in some embodiments, like the connector cable 104, the groundconductor 120 and the input conductor 122 may be considered to be partof the adaptive interface device 100.

When the person 124 touches the apple 126 (e.g., with a finger), acompleted circuit loop 128 forms between the keyboard input pin 116 g-wand the ground pin 116. The loop 128 includes the keyboard input pin 116g-w, the input conductor 122, the apple 126, the person 124, the groundconductor 120, and the ground pin 114. The completed circuit loop 128 isdetected by the microcontroller 106 of the PCB 102. Completing a circuitloop of one of the input pins 116, which is then detected by the PCB102, is an example of triggering the input pin 116. In response, the PCB102 sends the computer 112 an HID signal associated with the keyboardinput pin 116 g-w. That is, the PCB 102 generates an output thatsimulates a keyboard or mouse action (e.g., key press, mouse click, ormouse movement).

The computer 112 reacts to receiving the output from the PCB 102 as ifthe output were sent from a standard keyboard or mouse. Therefore, auser, such as the person 124 in the above-illustrated figure, cansimulate a key stroke on a keyboard by touching the apple 126, and thecomputer 112 will receive the simulated key stroke and react as if theuser had pressed the actual key on the key board that is beingsimulated. For instance, in a word processing program, and where the keystroke simulated is the letter “w,” upon the user touching the apple126, the word processing program would react as if a user pressed the“w” on the keyboard and display a new “w” on a display of the computer112.

The microcontroller 106 monitors the input pins 116 and uses highresistance switching and filtering to provide a sensitive detector thatsenses a completed circuit loop even through materials like skin,leaves, and modeling compound, which are not highly conductive. Forexample, the PCB 102 uses a pull-up resistor of twenty-two (22) megaohms. Software executing on the microcontroller 106 filters noise oneach input pin 116 using a moving window averager to lowpass filter.Alternatively, although potentially increasing costs, the adaptiveinterface device could use hardware filtering.

Although the apple 126 is illustrated above, any material that canconduct electricity, even if only slightly, will work to complete acircuit loop and be detected by the adaptive interface device 100. Otherexamples of conductive items used to complete a circuit loop between oneof the input pins 116 and the ground pins 114 of the PCB 102 includeketchup, pencil graphite, finger paint, lemons, plants, coins, otherhumans, silverware, water (and wet objects), most foods, cats, dogs,aluminum foil, rain, and many others.

Furthermore, if the person 124 is grounded, the input pins 116 aretouch-sensitive. In other words, rather than the person 124 touching theapple 126 to trigger the input 116 g-w, the user may directly touch theinput pin 116 g-w to complete the circuit loop.

Although described with respect to a word processing application, theadaptive interface device 100 similarly works with other programs andwebpages that take keyboard input, mouse input, and other HID input. Inanother example, the adaptive interface device 100 is used with acomputer program executing on the computer 112 that generates a virtualpiano. In typical operation, the virtual piano is played using keypresses on a keyboard and/or mouse actions. Instead of using thecomputer keyboard buttons to play the virtual piano, the adaptiveinterface device 100 generates outputs that are used to play the virtualpiano. Select ones of the input pins 116 can be connected to bananas viainput conductors (similar to the input conductor 122), the user toground via a ground conductor (similar to ground conductor 120), and thebananas become the piano keys. That is, each time a grounded usertouches one of the bananas, a circuit loop is completed between theground pin 114 and one of the input pins 116, which is detected by theadaptive interface device 100. In turn, the adaptive interface device100 generates an output to the computer 112 that simulates a key pressor mouse action, the particular key press or mouse action depending onthe particular one of the input pins 116 that is triggered. The outputis received by the computer program executing on the computer 112,resulting in playing a key or keys of the virtual piano.

As noted above, the adaptive interface device 100 is reprogrammable sothat the pin assignments of the input pins 116 can be changed. In otherwords, the adaptive interface device 100 can be reprogrammed to changethe output signal that is transmitted over the connector cable 104 inresponse to a particular one of the input pins 116 being triggered. Forinstance, in the example described with respect to FIG. 2, triggeringthe input pin 114 g-w causes a “W” HID code to be transmitted to thecomputer 112. However, after reprogramming, the adaptive interfacedevice 100 may be configured so that further triggering of the input pin114 g-w causes an “X” HID code to be transmitted instead of the “W” HDIcode.

In some embodiments, the reprogramming, also referred to as remapping,is carried out without having to connect the adaptive interface device100 to a computer having special drivers or specific software installedthereon. Moreover, the reprogramming is carried out withoutcommunicating firmware or other updates from a computer to the adaptiveinterface device 100. Rather, user inputs on the input pins 116 of theadaptive interface device 100 itself cause reprogramming of the adaptiveinterface device 100.

FIG. 3 illustrates a block diagram of reprogramming system 150 includingthe adaptive interface device 100, a local computer 154, and a remoteserver 156. The local computer 154 is a computing device and may be, forexample, a laptop, tablet, or desktop computer, such as the computer 112shown in FIG. 2. The adaptive interface device 100 includes a processor160, a memory 162, the input pins 116 (also referred to as programmableinputs), and the port 108. The processor 160 and memory 162 form part ofthe microcontroller 106 (shown in FIG. 1B) and the programmable inputsinclude the input pins 116 (shown in FIGS. 1A and 1B).

In some embodiments, the memory 162 includes instructions executed bythe processor 160, as well as data used by the processor 160, to carryout the functionality of the adaptive interface device 100 describedherein. The instructions include reprogramming code 170 executed by theprocessor 160 in a remapping mode, as well as other firmware 172, whichincludes at least code for a normal operation mode of the adaptiveinterface device. The reprogramming code 170 defines navigable datastructure 174 that are navigable by the user during reprogramming andcontrols the reprogramming of the input pins 116 in accordance with theuser's navigation.

In some embodiments, the memory 162 further includes a key mapping 180,which defines the correlation between (a) the input pins 116 of theadaptive interface device 152 that can be triggered by a user and (b)the correlating output codes that the adaptive interface device 152 isto generate in response to being triggered. For example, upon receipt bythe processor 160 of a user input in response to actuation of one of theinput pins 116, the processor 160 may access the memory 162 and use anidentifier of the actuated input pin 116 as an index into a data tableof the key mapping 180. The data table, also referred to as a keymapping index, associates the identifier with an HID output code, whichis provided back to the processor 160. The processor 160 then outputsthe returned HID output code on the port 108. The key mapping 180 mayinclude a current key mapping defining the current input pin-to-outputcode assignments, as well as a default key mapping that may be used tooverwrite the current key mapping upon a reset or restore operation ofthe adaptive interface device 100. In this example, to reprogram theadaptive interface device 100, at least a portion of the key mapping 180is overwritten or otherwise updated to change the stored inputpin-to-output code assignments. In other words, some or all of the inputpins 116 are remapped to different output codes.

In some embodiments, the local computer 154 is coupled to the remoteserver 156 via the Internet, which may include one or more wired and/orwireless connections. Although not illustrated, the local computer 154and remote server 156 each include a processor and memory. The localcomputer 154 includes a web browser software application (web browser)182 executed by the processor (not shown), a user interface including adisplay 184, and other user inputs (not shown), such as a keyboard andmouse, for interacting with the web browser 182. The user is operable toenter an address into or otherwise navigate the web browser 182 to a website for adaptive interface device reprogramming on the remote server156, which causes the remote server to transmit web pages including javaapplication software (web page software) 186 to the web browser 182. Theweb browser 182 interprets and/or executes the web page software 186,and provides corresponding visual output to the display 184, which, asdiscussed in further detail below, assists in reprogramming the adaptiveinterface device 100. In some embodiments, the local computer 154 andremote server 156 are coupled by a network connection not including theInternet, such as a local network or intranet connection. Additionally,in some embodiments, the web page software 186 is stored on the localcomputer 154 so that an Internet connection (or other connection) to theremote server 156 is not used during reprogramming of the adaptiveinterface device.

To reprogram, a user navigates the navigable data structure 174 withinthe microcontroller 106 of the adaptive interface device 100 throughcertain specified combinations of user inputs using the input pins 116.The adaptive interface device 100 as illustrated in FIGS. 1A and 1B doesnot itself have a display or simple means to provide visual feedback ofa user's reprogramming of the adaptive interface device. However, asdescribed above, the adaptive interface device 100 is able to outputsignals to the computer 154 to provide visual and/or audible feedback toa user to assist in programming. More particularly, as a user navigatesthe navigable data structure 174 to reprogram the adaptive interfacedevice 100, the processor 160 outputs signals representing thenavigation inputs received via the input pins 116 and generates outputsrepresenting the navigation inputs over the connector cable 104. Whilereprogramming, the web page software 186 is executing on the web browser182 of the coupled local computer 154, which is operable to receive theoutputs from the adaptive interface device 100 and provide the user witha graphical representation on the display 184 of the navigable datastructure 174 used to reprogram that the user is navigating.

To provide the graphical representation, the web browser 182 executingon the local computer 154 includes a corresponding copy of the navigabledata structure 174 received from the web page software 186, referred toas a replica data structure 188. The web page software 186 may includeseveral replica data structures 188, one for each type of adaptiveinterface device 100. The web browser 182 receives the outputs from theadaptive interface device 100 and navigates the replica data structure188 accordingly, providing real-time visual feedback of the user'snavigation. This browser-based graphical user interface provides theuser otherwise unavailable visual feedback and allows the user to moreeasily navigate an otherwise complex array of data structures within theadaptive interface device 100.

The communication between the adaptive interface device 100 and localcomputer 154 is one-way, from the adaptive interface device 100 to thelocal computer 154, i.e., to the web browser 182. As the communicationis one-way, the local computer 154 does not communicate back to theadaptive interface device 100, at least not substantively. In someinstances, certain acknowledgement and handshaking communications aresent from the local computer 154 to the adaptive interface device 100 toestablish or maintain a communication link according to certaincommunication protocols. In these instances, the communication betweenthe computer 154 and the adaptive interface device may still beconsidered one-way because substantive data payloads are not passed fromthe local computer 154 to the adaptive interface device 100. In otherwords, one-way communication means that substantive data payloads aretransmitted via a communication link in only one direction (e.g., fromdevice A to B, and not from device B to A), rather than in twodirections (e.g., from device A to B, and also from device B to A). Asan example, programming data and commands that update the key mapping180 (e.g., particular pin assignment values, save commands, and restorecommands) are not provided from the local computer 154. Rather, theprogramming data and commands are generated from within the adaptiveinterface device 100 by the reprogramming code 170 in response tonavigation input received by the input pins 116. Here, the programmingdata and commands are examples of substantive data payloads, while mereacknowledgement and handshaking communications to establish or maintaina communication link according to certain communication protocols arenonsubstantive.

During reprogramming, the web browser 182 is an application on the localcomputer 154 that is in the foreground to ensure that it receives thecommunication from the adaptive interface device 100. If the usernavigates to another application or webpage, causing the web browser 182to be in the background, and then the user triggers one of the inputpins 116 of the adaptive interface device 100, the web browser 182 maynot receive the user input. Therefore, synchronization between theactual navigation of the navigable data structure 174 on the adaptiveinterface device 100 and the replica data structure 188 on the webbrowser 182 may be lost. Thus, visual feedback of the user's actualnavigation on the adaptive interface device 100 from that point forwardmay be inaccurate.

In alternate instances, a user couples a five volt (5 volt) DC powersupply to the PCB 102, rather than connecting it to the local computer154 via the connector cable 104, and programs the adaptive interfacedevice 100 without visual feedback from the local computer 154.Additionally, the adaptive interface device 100 is but an example of theinterface devices that may be programmed as described herein. Otherinterface devices may use include other techniques for receiving userinputs and triggering inputs that result in outputs of the interfacedevice. Such other techniques may include one or more of pushbuttons,keypads, optical sensors, and capacitive sensors interfacing with amicrocontroller.

FIGS. 4A and 4B illustrate methods 200 and 210, respectively, forprogramming an adaptive interface device. The methods 200 and 210 aredescribed with respect to reprogramming the adaptive interface device100; however, in some embodiments, the methods are used to reprogramother interface devices. The method 200 is generally described from theperspective of a computing device that is coupled to an adaptiveinterface device being reprogrammed, while method 210 is generallydescribed from the perspective of an adaptive interface device beingreprogrammed. Together, the methods 200 and 210 may be carried out bycomponents of a reprogramming system, such as reprogramming system 150,to reprogram an adaptive interface device.

In step 220 of the method 200 of FIG. 4A, a remapping graphical userinterface (remapping GUI) 221 is provided on the display 184 of thelocal computer 154. The remapping GUI 221 may be provided by the webbrowser 182 based on the web page software 186 obtained from the remoteserver 156, as described above, or based on a local software applicationresiding on the local computer 154. The remapping GUI 221 includesvarious screens to convey information to a user, examples of which areshown and described with respect to FIGS. 5A-5E.

For instance, in step 220, a user starts the web browser 182 on thelocal computer 154, which is connected to the Internet, and navigates tothe reprogramming web page having the web page software 186. The webbrowser 182, in turn, displays a remapping start screen 222 illustratedin FIG. 5A. As shown below, this screen 222 instructs the user todisconnect their adaptive interface device 100 and to then initiate theprocess by clicking a start button 224 (e.g., using a mouse coupled tothe local computer 154). This clicking action ensures that the webbrowser 182 is in the foreground on the local computer 154 and will bereceiving data from the adaptive interface device 100 in later stages.

Upon selecting the start button 224, the remapping GUI 221 is updated todisplay the enter remap mode screen 226 (FIG. 5B). The screen 226instructs the user how to boot the adaptive interface device 100 in theremap mode. In the example instructions on the screen 226, the user isinstructed to connect the input pin 116 a and 116 b (e.g., with a firstalligator clip cable), to connect the input pin 116 c and 116 d (e.g.,with a second alligator clip cable), and then to connect the adaptiveinterface device 100 to the computer 154 using the connector cable 104.When the adaptive interface device 100 is booted in the remap mode,light emitting diodes (LEDs) of the PCB 102 will slowly pulse on and offto indicate to the user that the boot was successful and that theadaptive interface device 100 is in the remap mode, not in the normaloperating mode.

After the user has followed the instructions set out in the screen 226,the local computer 154 receives a remap mode message from the adaptiveinterface device 100 (step 228). For example, the adaptive interfacedevice 100 outputs a confirmation string upon entering the remap mode,the string including a name of the adaptive interface device 100, thesoftware version, and the hardware version (e.g., “mm v1.20ab”). The webbrowser 182 can determine from the confirmation string the software andhardware version of the coupled adaptive interface device 100.Accordingly, when the adaptive interface device 100 is revised or analternate version is coupled that may have different layouts, inputs,and capabilities, the web browser 182 can proceed appropriately (e.g.,obtain the appropriate replica data structure 188).

In response to receiving the remap mode message, the web browser 182advances the remapping GUI 221 to a confirmation screen 230 of FIG. 5C.The confirmation screen 230 instructs the user to disconnect thealligator clips. When the adaptive interface device 100 detects that thealligator clips are disconnected, the adaptive interface device 100sends current mapping data to the web browser 182 (step 232). Forexample, the current mapping data may include a fifty-seven characterconfiguration string to the web browser 182 that identifies the currentprogramed mapping of the adaptive interface device 100 (i.e., the keymapping 180), such as“suph50h52h51h4fh2chf0h1ah04h16h07h09h0ahf4hf5hf2hf3hf0hf1,” plus threeadditional characters (e.g., “h50”) indicating the position that theuser is located at within the navigable data structure 174.

After receipt of the current mapping data, the web browser 182 advancesthe remapping GUI 221 to a remapping screen 236, as illustrated on FIG.5D (in step 234). The remapping screen 236 includes display of agraphical representation of the navigable data structure 174. Thegraphical representation of the navigable data structure 174 includes agraphic representation of the adaptive interface device 100 and itsinput pins 116 (virtual adaptive interface 240), an onscreen keyboard(OSK) 242 (also referred to as a virtual keyboard 242), and virtualprogramming control buttons 244.

The virtual adaptive interface 240 that is shown is based on the versioninformation transmitted as part of the remap mode message in step 228.In other words, the particular shape, layout, and programmable inputsillustrated are based on the version information. Additionally, thevirtual adaptive interface 240 is shown with the current mapping of theinput pins 116 provided with the current mapping data in step 232.Accordingly, the virtual adaptive interface 240 is a visualrepresentation of the front and back sides of the adaptive interfacedevice 100 with current mappings for each of the input pins 116.

The virtual programming control buttons 244 include a row having a savebutton 246, a cancel button 247, and a restore button 248, which aredescribed in further detail below.

Navigation of the remapping screen 236 is performed by way of userinputs on the adaptive interface device 100, which are interpreted andconveyed as navigation codes over the one-way communication to the localcomputer 154. The user may navigate the remapping screen 236 by causingactuation of (triggering) the up arrow input pin 116 a, down arrow inputpin 116 b, left arrow input pin 116 c, and right arrow input pin 116 d,and the click input pin 116 f on the adaptive interface device 100. Asnoted, navigating the remapping screen 236 through triggering theseinput pins 116 visualizes the actual navigation of data structure 174 onthe adaptive interface device 100 that is occurring through the sametriggering. Triggering one of these input pins 116 generates anavigation control code that is sent to the web browser 182 over theconnector cable 104. The navigation control code is a three characterhex code starting with an “x” value. More specifically, the navigationcontrols codes are set forth in TABLE I as follows:

TABLE I Navigation Control Codes (X_ _) Up X52 Right X4F Down X51 ClickXF0 Left X50

Navigation of the remapping screen 236, including the virtual adaptiveinterface 240, virtual keyboard 242, and virtual programming controlbuttons 244, is linear. A cursor 250 is provided on the remapping screen236. The cursor 250 may be a contrasting color, flashing element, or, asillustrated, a circle, that highlights a current position of the userwithin the navigable data structure 174.

The adaptive interface device 100 receives various navigation inputsfrom the user triggering input pins 116 and, in response to eachnavigation input, the adaptive interface device 100 outputs a navigationcode to the web browser 182 of the local computer 154 (step 252). Theweb browser 182, in turn, updates the remapping screen 236 in accordancewith the received navigation codes (step 254). The updates are, forinstance, changing the location of the cursor 250 on the remappingscreen 236. After updating the remapping screen 236 based on navigationcodes, the web browser 182 determines whether the most recent navigationcode indicates that the adaptive interface device 100 has been remapped(e.g., the save button 246 has been selected). If the adaptive interfacedevice 100 has not been remapped, the method 200 returns to step 252.Steps 252 and 254 may be repeated as a user navigates the navigable datastructure 174 and step 256 continues to be evaluated to be false.

For example, when the cursor 250 is on the virtual adaptive interface240, the right arrow navigation codes cause the cursor 250 to proceedthrough the input pins 116 in the following order: left arrow, up arrow,down arrow, right arrow, space, click, W, A, S, D, F, G, mouse up, mousedown, mouse left, mouse right, mouse left click, mouse right click, andthen, looping back, the left arrow again.

Triggering the down arrow input pin 114 b on the adaptive interfacedevice 100 will drop the cursor off of the virtual adaptive interface240 and to the virtual programming control buttons 244, where the usercan select the save button 246, cancel button 247, or restore button248. Triggering the up arrow input pin 114 a on the adaptive interfacedevice 100 brings the cursor 250 back up to the virtual adaptiveinterface 240.

Navigation to the virtual keyboard 242 is performed by triggering theclick input pin 116 f on the adaptive interface device 100 when thecursor 250 is on one of the (virtual) input pins 116 of the virtualadaptive interface 240. Selecting one of the (virtual) input pins 116and navigating to the virtual keyboard 242 allows the user to modify thekey/control assigned to the selected one of the input pins 116 of theadaptive interface device 100. Upon triggering the click input pin 116 fwhen the cursor 250 is on one of the virtual input pins 116, the cursor250 jumps to the location of the currently assigned key on the virtualkeyboard 242. The user can then navigate to a new key on the virtualkeyboard 242 by triggering the arrow input pins 114 a-d on the adaptiveinterface device 100 to control the cursor 250 on the virtual keyboard242. By triggering the click input pin 116 f again, the cursor 250returns to the virtual input pin 116 on the virtual adaptive interface240. The selected virtual input pin 116 is also graphically changed tothe new key previously highlighted on the virtual keyboard 242 when theclick input pin 116 f was triggered. For example, FIG. 5E illustratesthe remapping screen 236 after the input pin 116 g-w on the virtualadaptive interface 240 is updated from a “W” to an “X” character.

While the virtual adaptive interface 240 now has a modified key mapping,the actual key mapping 180 of the adaptive interface device 100 is notyet updated. Rather, to update the key mapping 180 with the modified keymapping displayed on the virtual interface device 240, a user positionsthe cursor 250 on the save button 246 and triggers the click input pin116 f. Upon triggering the click input pin 116 f when the cursor 250 ispositioned on the save button 246, the adaptive interface device 100 isremapped. This remapping is detected in step 256 by the web browser 182based on the navigation inputs, including the triggering of the clickinput pin 116 f when the cursor 250 was positioned on the save button246.

In response to detecting the remapping, the web browser 182 displays theupdated key mapping 180 of the adaptive interface device 100 on thevirtual adaptive interface 240. In some instances, the virtual adaptiveinterface 240 will already be showing the updated key mapping uponselection of the save button 246 and, in step 258, the virtual interfacedevice 240 is unchanged. In some instances, in step 258, an additionalvisual indicator (e.g., flash, color change, or particular text) isshown on the remapping screen 236 to indicate that the mapping has beenupdated.

In step 260, the web browser 182 determines whether additionalnavigation input is received. If additional navigation codes arereceived by the web browser 182, the method returns to step 254 toupdate the remapping screen 236. If no additional navigation codes arereceived, e.g., for a predetermined amount of time, the method 200 ends.The method 200 may also be exited through selection of the cancel button247.

An example of reprogramming the input pin 116 g-w using the method 200is now provided. To reprogram the input pin 116 g-w of the adaptiveinterface device 100 to output an “X” value instead of a “W,” the usercan take the following steps:

-   -   i. navigate the cursor 250 to the “W” (input pin 116 g-w) of the        virtual adaptive interface 240 and trigger the click input pin        116 f on the adaptive interface device 100, causing the cursor        250 to jump down to the “W” on the virtual keyboard 242;    -   ii. position the cursor on the “X” of the virtual keyboard 242        by actuating the right arrow input pin 116 d once on the actual        adaptive interface device 100;    -   iii. trigger the click input pin 116 f, causing the cursor 250        to return to the input pin 116 g-w on the virtual adaptive        interface 240, which is then graphically changed from having a        “W” to an “X” (see FIG. 5E);    -   iv. trigger the down arrow input pin 116 b on the adaptive        interface device to move the cursor 250 to the save button 246;    -   v. trigger the click input pin 116 f to save the new mapping on        the adaptive interface device

As noted previously, selecting the save button 246 does not cause theupdated mapping to be sent from the computer 154 to the adaptiveinterface device 100. Rather, the various navigation inputs provide bythe user in the steps i., ii., and iii. to the adaptive interface device100 are received by the reprogramming code 170 and, when the save button246 is selected in step v., the reprogramming code 170 updates the keymapping 180 according to the received user inputs.

The cancel button 247, when selected by the user, is used to cause theadaptive interface device 100 to exit the remapping mode and return tothe normal operating mode. Upon returning to the normal operating mode,the key mapping 180 will have the values assigned in the most recentsave operation.

The restore button 248, when selected by the user, is used to cause theadaptive interface device 100 to return to default settings. Uponselecting the restore button 248, the key mapping 180 is overwrittenwith default key assignments.

Returning to FIG. 4B, the method 210 for programming of the adaptiveinterface device 100 is illustrated. The method 210 begins with theadaptive interface device 100 receiving inputs that cause entry into theremapping mode (step 270). More particularly, as described above withrespect to FIG. 5B, a user may connect particular input pins 116together and then connect the adaptive interface device 100 to the localcomputer 154 via the connector cable 104 to provide power thereto. Uponproviding power to the adaptive interface device 100, the particularconnections are detected by the microcontroller 106, and the adaptiveinterface device 100 enters into the remapping mode.

After the adaptive interface device 100 enters the remapping mode, theadaptive interface device 100 transmits the remap mode message to thelocal computer 154 (step 272). The remap mode message is transmitted instep 272 as it was described with respect to step 228 of FIG. 4A. Instep 274, upon detecting that the particular connections of the inputpins 116 used to enter the remapping mode have been broken, the adaptiveinterface device 100 transmits the current mapping data to the localcomputer 154, as it was described above with respect to step 232 of FIG.4A.

In step 276, the adaptive interface device 100 receives navigation inputat the input pins 116. In other words, the user triggers various inputpins 116 by selectively connecting the input pins 116 to ground. Basedon the receiving navigation inputs, in step 278, the navigable datastructure 178 is navigated and the adaptive interface device 100 isremapped. For instance, the navigation inputs cause the navigation ofnavigable data structure 174, which leads the reprogramming code 170 toupdate the key mapping 180, as described in greater detail above.

In step 280, the adaptive interface device 100 transmits navigationcodes to the web browser 182 of the local computer 154 indicative of thenavigation input received. In turn, the web browser 182 updates theremapping screen 236 (see FIG. 5D). In practice, the steps 276, 278, and280 are repeatedly cycled between and overlap in execution during theprocess of remapping the programmable inputs of the adaptive interfacedevice 100. Particularly, for each navigation input received in step276, a navigation code is transmitted in step 280, and the transmissionsof step 280 occur in real time as navigation inputs are received in step276. This overlapping execution of steps 276, 278, and 280 contrastswith sequentially carrying out step 276, 278, and then step 280 bywaiting for step 276 to be complete before proceeding to step 278 andwaiting for step 278 to be complete before proceeding to step 280. Thenavigation codes transmitted in step 280 are used to update theremapping screen 236 as described above with respect to method 200 ofFIG. 4A. In step 282, if further navigation input is received by theadaptive interface device 100, the method returns to step 278. If noadditional navigation codes are received in step 282, e.g., for apredetermined amount of time, the method 210 ends. The method 210 mayalso be exited through selection of the cancel button 247.

The adaptive interface device 100 and the remapping screen 236 of theweb browser 182 are semi-independent state machines. The state of theremapping screen 236 is controlled by the adaptive interface device 100,but the adaptive interface device 100 is not controlled by the remappingscreen 236 and does not receive data from the remapping screen 236 toconfirm that the state machines are in synchronization. Therefore, inaddition to the navigation control codes (L, R, U, D, Click), theadaptive interface device 100 sends a navigation state code (i.e., theexpected location of the cursor 250) with each input pin 116 actuation.Each navigation state code is a three-character hex code as well, butthe navigation state codes start with an “h” while the navigationcontrol codes start with an “x.” For instance, the up arrow navigationstate code is represented by “h52,” while the up arrow navigationcontrol code is represented by an “x52.” A list of navigation statecodes is provided below in Table II.

TABLE II Navigation State Codes (H_ _) Letters: A (H04) to Z (H1D)Backspace H2A Numbers: 1 (H1E) to 0 (H27) Esc H29 Space H2C Up H52 EnterH28 Down H51 Tab H2B Left H50 ’ H36 Right H4F . H37 Mouse up HF4 / H38Mouse down HF5 ; H33 Mouse Left HF2 ‘ H34 Mouse right HF3 [ H2F Mouseleft click HF0 ] H30 Mouse right click HF1 \ H31 Volume Mute H7F ′ H35Volume Down H81 H2D Volume Up H80 = H2E

For example, as noted above, when the current mapping data is providedto the web browser 182 in step 232, three additional characters (e.g.,“h50”) indicating the current position within the navigable datastructure 174 (i.e., the location of the cursor 250). Accordingly, thecursor 250 starts at the left arrow on the virtual adaptive interface240 based on the coding of TABLE II above. When the user presses theright arrow (input pin 116 d) on adaptive interface device 100, thestring “x4fh52” is sent as the navigation code, which includes thenavigation control code “x4f” and the navigation state code “h52.” Thisnavigation code indicates to the web browser 182 that (1) the cursor 250is to be shifted to the right one increment based on the navigationcontrol code “x4f” and (2) that the expected location of the cursor 250after the shift is the up arrow based on the navigation state code“h52.” Pressing the right arrow again causes the adaptive interfacedevice to output the string “x4fh51” as the navigation code, whichindicates to advance the cursor 250 one increment right (“x4f”) andconfirms that the cursor 250 will then be located at the down arrow(“h51”).

Since the communication between the adaptive interface device 100 andthe local computer 154 is one way and the web browser 182 does notcommunicate back to the adaptive interface device 100, user navigationsof” the remapping screen 236 through an input device other than theadaptive interface device 100 would remain unknown to the adaptiveinterface device 100. Accordingly, the remapping GUI 221 and web browser182 is set such that computer inputs via other typical peripherals(e.g., keyboard or mouse) do not effect or navigate the remapping screen236. This technique assists in allowing the remapping screen 236 to staysynchronized with the internal state of the adaptive interface device100.

To ensure that the user cannot use a standard keyboard or mouse tonavigate the remapping screen 236, the navigation control codes that theadaptive interface device 100 outputs are non-standard hex strings,instead of the standard HID equivalent. For instance, while in a normaloperation mode, the user may trigger the right arrow input pin 116 d onthe adaptive interface device 100 to output an HID equivalent thatnavigates right on the display 184 of the local computer 154, when inremapping mode, the right arrow on the adaptive interface device 100actually outputs “x4f” (along with the three character hex coderepresenting the expected state), which are not standard HID codes.Thus, if the user pressed the right arrow on a keyboard connected to thelocal computer 154, the remapping screen 236 would not recognize theinput.

If the local computer 154 fails to receive a user input for apredetermined amount of time while in the remapping mode, the webbrowser will display a time out message. The time out message mayindicate that communication has timed out and request that the userrefresh the page to start over (e.g., at the screen 222 of FIG. 5A). Theadaptive interface device 100 may also be setup to exit the remappingmode if it fails to receive a user input for the predetermined amount oftime while in the remapping mode. If such an automatic exit occursbecause of a time out period being reached, the adaptive interfacedevice 100 may resort back to default programming of its input pins 116,or may resort back to the most recently saved programming of its inputpins 116.

The embodiments of the adaptive interface device 100 described above areexamples of devices that are reprogrammable using the above-notedtechniques. In some embodiments, the reprogramming techniques are usedto reprogram, for example, other programmable computer interface devicesand/or other programmable devices that do not generally serve as acomputer interface device. For example, a programmable device that isnot generally a computer interface device may include a PCB with amicrocontroller mounted thereon and various inputs and outputs (e.g.,pins, LEDs, sensors, vibration generators, and/or speakers) usable forvarious functions that, during typical operation, are independent ofinterfacing with a computer. In a reprogramming mode, however, theprogrammable device is coupled to a computer, e.g., using a USB cable,and the computer then provides real time visual feedback of the user'sreprogramming based on one-way communication using similar techniques asdescribed above. For instance, the reprogramming may be used to altersettings of the microcontroller, alter input or output parameters, remapinputs and/or outputs to be assigned to different values or functions,restore default settings, or customize other programmable aspects of thedevice.

Thus, the invention provides, among other things, an adaptive interfacedevice that is programmable and a system and method of programming anadaptive interface device. Various features and advantages of theinvention are set forth in the following claims.

What is claimed is:
 1. A method of programming an adaptive interfacedevice, the method comprising: providing a remapping graphical userinterface on a display screen of a computing device; providing, on theremapping graphical user interface, a graphical representation of anavigable data structure of the adaptive interface device; receiving,from the adaptive interface device, navigation codes; updating thegraphical representation based on the navigation codes; determining,based on the navigation codes, that the adaptive interface device hasbeen remapped to have an updated mapping; and displaying, on theremapping graphical user interface, the updated mapping of the adaptiveinterface device.
 2. The method of claim 1, further comprising:receiving, by the computing device, a data cable coupled to the adaptiveinterface device, wherein the navigation codes are received over thedata cable.
 3. The method of claim 1, further comprising: receiving, aremap mode message from the adaptive interface device, the remap modemessage indicating that the adaptive interface device has entered aremap mode.
 4. The method of claim 1, further comprising: receiving,from the adaptive interface device, current mapping data; anddisplaying, on the remapping graphical user interface, a current mappingof the adaptive interface device based on the current mapping data. 5.The method of claim 1, further comprising: displaying, on the remappinggraphical user interface, a virtual adaptive interface device includingprogrammable inputs of the adaptive interface device.
 6. The method ofclaim 5, wherein displaying the updated mapping of the adaptiveinterface device includes displaying the updated mapping on the virtualadaptive interface device.
 7. The method of claim 1, wherein thegraphical representation of the navigable data structure is a virtualkeyboard.
 8. A method of programming an adaptive interface device, themethod comprising: transmitting, by the adaptive interface device, aremap mode message to a computing device, the remap mode messageindicating that the adaptive interface device has entered a remap mode;receiving, by the adaptive interface device, navigation input thatnavigates a navigable data structure of the adaptive interface deviceused to remap programmable inputs of the adaptive interface device;remapping, by the adaptive interface device, the programmable inputsbased on the navigation input; and transmitting, by the adaptiveinterface device, navigation codes indicative of the navigation inputreceived and indicative of the remapping.
 9. The method of claim 8,further comprising: receiving, by the adaptive interface device, a datacable coupled to the computing device, wherein the navigation codes aretransmitted over the data cable.
 10. The method of claim 8, furthercomprising: upon entering the remap mode, transmitting current mappingdata to the computing device, the current mapping data indicative of acurrent mapping of the programmable inputs of the adaptive interfacedevice.
 11. The method of claim 8, further comprising: receiving arequest, via the programmable inputs, to enter the adaptive interfacedevice into the remap mode.
 12. The method of claim 11, furthercomprising: before receiving the request to enter the remap mode,receiving user input at a first programmable input of the programmableinputs; transmitting, by the adaptive interface device, a first code tothe computing device representing a first mapped output in response tothe user input; receiving further user input at the first programmableinput after the remapping; and transmitting, by the adaptive interfacedevice, a second code to the computing device representing a secondmapped output in response to the further user input.
 13. An adaptiveinterface device that is programmable, the adaptive interface devicecomprising: programmable inputs; an input/output interface configured tobe coupled to a computing device; a memory including a navigable datastructure and a key mapping index; and a processor configured totransmit a remap mode message via the input/output interface to thecomputing device, the remap mode message indicating that the adaptiveinterface device has entered a remap mode; receive, via the programmableinputs, navigation input that navigates the navigable data structure;remap the programmable inputs based on the navigation input; andtransmit, via the input/output interface to the computing device,navigation codes indicative of the navigation input received andindicative of the remapping.
 14. The programmable adaptive interfacedevice of claim 13, further comprising: a data cable coupled to theinput/output interface and the computing device to form a communicationlink.
 15. The programmable adaptive interface device of claim 13,wherein the key mapping index relates each of the programmable inputs toa corresponding output code.
 16. The programmable adaptive interfacedevice of claim 13, wherein, upon entering the remap mode, the keymapping index has current mapping data indicative of a current mappingof the programmable inputs of the adaptive interface device.
 17. Theprogrammable adaptive interface device of claim 13, wherein uponcompletion of the remapping, the key mapping index has updated mappingdata indicative of an updated mapping of the programmable inputs of theadaptive interface device.
 18. The programmable adaptive interfacedevice of claim 13, wherein the processor is further configured to,receive user input at a first programmable input of the programmableinputs; transmit a first code to the computing device representing afirst mapped output in response to the user input; receive further userinput via the first programmable input after the remapping; and transmita second code to the computing device representing a second mappedoutput in response to the further user input.
 19. The programmableadaptive interface device of claim 13, further comprising: a conductorselectively secured to two of the programmable inputs to form aconductive path between the two programmable inputs, and wherein theprocessor is further configured to detect the conductive path formed bythe conductor and, in response, enter the adaptive interface device intothe remap mode.